License

This document is licensed under a Creative Commons Attribution 4.0 International License.

The code is licensed under the MIT License.

The data used is licensed under a Open Data Commons Attribution License.

See the paper’s “Author Contributions” section for details on the contributors of data files.

Metadata

Required libraries and runtime environment description.

## - Session info ----------------------------------------------------------
##  setting  value                       
##  version  R version 3.6.1 (2019-07-05)
##  os       Windows 10 x64              
##  system   x86_64, mingw32             
##  ui       RTerm                       
##  language (EN)                        
##  collate  Spanish_Spain.1252          
##  ctype    Spanish_Spain.1252          
##  tz       Europe/Paris                
##  date     2019-11-25                  
## 
## - Packages --------------------------------------------------------------
##  ! package     * version    date       lib source        
##    assertthat    0.2.1      2019-03-21 [1] CRAN (R 3.6.0)
##    backports     1.1.4      2019-04-10 [1] CRAN (R 3.6.0)
##    base        * 3.6.1      2019-07-05 [?] local         
##    broom         0.5.2      2019-04-07 [1] CRAN (R 3.6.0)
##    callr         3.3.0      2019-07-04 [1] CRAN (R 3.6.1)
##    cellranger    1.1.0      2016-07-27 [1] CRAN (R 3.6.0)
##    cli           1.1.0      2019-03-19 [1] CRAN (R 3.6.0)
##    colorspace    1.4-1      2019-03-18 [1] CRAN (R 3.6.0)
##  P compiler      3.6.1      2019-07-05 [2] local         
##    cowplot     * 1.0.0      2019-07-11 [1] CRAN (R 3.6.1)
##    crayon        1.3.4      2017-09-16 [1] CRAN (R 3.6.0)
##  P datasets    * 3.6.1      2019-07-05 [2] local         
##    desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.1)
##    devtools      2.1.0      2019-07-06 [1] CRAN (R 3.6.1)
##    digest        0.6.19     2019-05-20 [1] CRAN (R 3.6.0)
##    dplyr       * 0.8.2      2019-06-29 [1] CRAN (R 3.6.0)
##    DT          * 0.7        2019-06-11 [1] CRAN (R 3.6.1)
##    evaluate      0.14       2019-05-28 [1] CRAN (R 3.6.0)
##    forcats     * 0.4.0      2019-02-17 [1] CRAN (R 3.6.0)
##    fs            1.3.1      2019-05-06 [1] CRAN (R 3.6.0)
##    generics      0.0.2      2018-11-29 [1] CRAN (R 3.6.0)
##    ggplot2     * 3.2.0      2019-06-16 [1] CRAN (R 3.6.0)
##    glue          1.3.1      2019-03-12 [1] CRAN (R 3.6.0)
##  P graphics    * 3.6.1      2019-07-05 [2] local         
##  P grDevices   * 3.6.1      2019-07-05 [2] local         
##  P grid          3.6.1      2019-07-05 [2] local         
##    gt.emidata  * 0.0.0.9000 2019-11-22 [1] local         
##    gtable        0.3.0      2019-03-25 [1] CRAN (R 3.6.0)
##    haven         2.1.0      2019-02-19 [1] CRAN (R 3.6.0)
##    here        * 0.1        2017-05-28 [1] CRAN (R 3.6.1)
##    hms           0.4.2      2018-03-10 [1] CRAN (R 3.6.0)
##    htmltools     0.3.6      2017-04-28 [1] CRAN (R 3.6.0)
##    htmlwidgets   1.3        2018-09-30 [1] CRAN (R 3.6.1)
##    httr          1.4.0      2018-12-11 [1] CRAN (R 3.6.0)
##    jsonlite      1.6        2018-12-07 [1] CRAN (R 3.6.0)
##    kableExtra  * 1.1.0      2019-03-16 [1] CRAN (R 3.6.1)
##    knitr       * 1.23       2019-05-18 [1] CRAN (R 3.6.0)
##    lattice       0.20-38    2018-11-04 [2] CRAN (R 3.6.1)
##    lazyeval      0.2.2      2019-03-15 [1] CRAN (R 3.6.0)
##    lubridate     1.7.4      2018-04-11 [1] CRAN (R 3.6.0)
##    magrittr      1.5        2014-11-22 [1] CRAN (R 3.6.0)
##    memoise       1.1.0      2017-04-21 [1] CRAN (R 3.6.1)
##  P methods     * 3.6.1      2019-07-05 [2] local         
##    modelr        0.1.4      2019-02-18 [1] CRAN (R 3.6.0)
##    munsell       0.5.0      2018-06-12 [1] CRAN (R 3.6.0)
##    nlme          3.1-140    2019-05-12 [2] CRAN (R 3.6.1)
##    pillar        1.4.2      2019-06-29 [1] CRAN (R 3.6.0)
##    pkgbuild      1.0.3      2019-03-20 [1] CRAN (R 3.6.1)
##    pkgconfig     2.0.2      2018-08-16 [1] CRAN (R 3.6.0)
##    pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.6.1)
##    prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.6.0)
##    processx      3.4.0      2019-07-03 [1] CRAN (R 3.6.1)
##    ps            1.3.0      2018-12-21 [1] CRAN (R 3.6.0)
##    purrr       * 0.3.2      2019-03-15 [1] CRAN (R 3.6.0)
##    R6            2.4.0      2019-02-14 [1] CRAN (R 3.6.0)
##    Rcpp          1.0.1      2019-03-17 [1] CRAN (R 3.6.0)
##    readr       * 1.3.1      2018-12-21 [1] CRAN (R 3.6.0)
##    readxl        1.3.1      2019-03-13 [1] CRAN (R 3.6.0)
##    remotes       2.1.0      2019-06-24 [1] CRAN (R 3.6.1)
##    rlang         0.4.0      2019-06-25 [1] CRAN (R 3.6.0)
##    rmarkdown     1.13       2019-05-22 [1] CRAN (R 3.6.0)
##    rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.6.0)
##    rstudioapi    0.10       2019-03-19 [1] CRAN (R 3.6.0)
##    rvest         0.3.4      2019-05-15 [1] CRAN (R 3.6.0)
##    scales      * 1.0.0      2018-08-09 [1] CRAN (R 3.6.0)
##    sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.1)
##  P stats       * 3.6.1      2019-07-05 [2] local         
##    stringi       1.4.3      2019-03-12 [1] CRAN (R 3.6.0)
##    stringr     * 1.4.0      2019-02-10 [1] CRAN (R 3.6.0)
##    testthat      2.1.1      2019-04-23 [1] CRAN (R 3.6.1)
##    tibble      * 2.1.3      2019-06-06 [1] CRAN (R 3.6.0)
##    tidyr       * 0.8.3      2019-03-01 [1] CRAN (R 3.6.0)
##    tidyselect    0.2.5      2018-10-11 [1] CRAN (R 3.6.0)
##    tidyverse   * 1.2.1      2017-11-14 [1] CRAN (R 3.6.1)
##  P tools         3.6.1      2019-07-05 [2] local         
##    usethis       1.5.1      2019-07-04 [1] CRAN (R 3.6.1)
##  P utils       * 3.6.1      2019-07-05 [2] local         
##    viridisLite   0.3.0      2018-02-01 [1] CRAN (R 3.6.0)
##    webshot       0.5.1      2018-09-28 [1] CRAN (R 3.6.1)
##    withr         2.1.2      2018-03-15 [1] CRAN (R 3.6.0)
##    xfun          0.8        2019-06-25 [1] CRAN (R 3.6.0)
##    xml2          1.2.0      2018-01-24 [1] CRAN (R 3.6.0)
##    yaml          2.2.0      2018-07-25 [1] CRAN (R 3.6.0)
## 
## [1] C:/Users/Carlos Granell/Documents/R/win-library/3.6
## [2] C:/Program Files/R/R-3.6.1/library
## 
##  P -- Loaded and on-disk path mismatch.

This document is versioned in a public git repo: https://github.com/cgranell/apps-treatment-mental-disorders.

This R notebook contains the code to produce the final figures and tables of the paper.

Data

The final number of surveyed papers is 158. For each paper, we have extracted 68 items. Each item is described in the methods section of the paper.

About 71.5% (N=113) of papers reported some kind of assessment, whereas 28.5% (N=45) reported no assessment at all. Looking closer at the type of assessment: 13.9% (N=22) focus on the effect of intervention on clinical symptomology; 32.3% (N=51) report a mix of effect and usability / UX assessment; 25.3% (N=40) focus solely on usability / UX.

Figures and tables

Figure 2: Temporal trend of the number of articles published and assessment type.

It combines two plots. The top one is a stacked bar chart over years, and the value of each group (assessment type) is in absolute counts. The bottom one is a proportional stacked area chart in which the sum of each year is always equal to hundred, and the value of each group (assessment type) is in percentages.

NEW TABLE: Key variables (sample, experimental conditions, resutls, etc) of ‘RCT’ or ‘pilot RCT’ studies only.

NOTE: In response to the comment G.26 ‘Characteristics of incldued studies’.

NEW TABLE: Evolution of assessment types

NOTE: In response to the comment G.18 “The authors may consider presenting the data in the ‘Evolution of research and assessment’ paragraph using a table instead”. The following table can simplify the last paragragh of the section ‘Evolution of research and assessment’

Distribution of assessment type
Assessment type N (%) RCT - N (%) PILOT RCT - N (%)
NO ASSESSMENT 45 (28.5%) 1 (0.6%) 0 (0%)
USABILITY/UX 40 (25.3%) 2 (1.3%) 0 (0%)
EFFECT AND USABILITY/UX 51 (32.3%) 15 (9.5%) 5 (3.2%)
EFFECT 22 (13.9%) 9 (5.7%) 2 (1.3%)

Figure 3: Distribution of articles per mental disorder, categorized according to assessment type. The embedded pie chart shows the proportional distribution of types of assessment over all mental disorders.

The main plot is a stacked barchart colored by assessment type along with a lollipop chart (white circle at the end of each bar) that shows the total percentatge of each mental disorder. Note that we draw two main colors in the legend: purple represents no assessment at all, the range of greens denotes distinct types of assessment. A pie chart is embedded to show the proportional distribution of types of assessment over all mental disorders.

To complement the previous figure, some percentages are discussed in the section ‘Relation between assessment types and mental disorder’ which are included in the following.

Mental Disorder Type of assessment Number of studies Relative %
Depressive disorders EFFECT 6 19.4%
EFFECT AND USABILITY/UX 12 38.7%
NO ASSESSMENT 9 29%
USABILITY/UX 4 12.9%
Various disorders EFFECT 2 9.5%
EFFECT AND USABILITY/UX 7 33.3%
NO ASSESSMENT 4 19%
USABILITY/UX 8 38.1%
Anxiety disorders EFFECT 5 27.8%
EFFECT AND USABILITY/UX 4 22.2%
NO ASSESSMENT 8 44.4%
USABILITY/UX 1 5.6%
Substance-related and addictive disorders EFFECT 2 12.5%
EFFECT AND USABILITY/UX 5 31.2%
NO ASSESSMENT 8 50%
USABILITY/UX 1 6.2%
Schizophrenia spectrum and other psychotic disorders EFFECT AND USABILITY/UX 4 26.7%
NO ASSESSMENT 5 33.3%
USABILITY/UX 6 40%
Trauma and stressor-related disorders EFFECT 2 13.3%
EFFECT AND USABILITY/UX 7 46.7%
NO ASSESSMENT 1 6.7%
USABILITY/UX 5 33.3%
Suicidal behavior disorder/nonsuicidal self-injury EFFECT 1 9.1%
EFFECT AND USABILITY/UX 5 45.5%
NO ASSESSMENT 3 27.3%
USABILITY/UX 2 18.2%
Comorbid disorders EFFECT 1 14.3%
NO ASSESSMENT 2 28.6%
USABILITY/UX 4 57.1%
Bipolar and related disorders EFFECT 1 16.7%
EFFECT AND USABILITY/UX 2 33.3%
USABILITY/UX 3 50%
Obsessive-Compulsive and related disorders EFFECT 1 20%
EFFECT AND USABILITY/UX 3 60%
USABILITY/UX 1 20%
Neurodevelopmental disorders EFFECT AND USABILITY/UX 1 25%
NO ASSESSMENT 1 25%
USABILITY/UX 2 50%
Feeding and eating disorders NO ASSESSMENT 1 25%
USABILITY/UX 3 75%
Sleep-wake disorders EFFECT 1 33.3%
NO ASSESSMENT 2 66.7%
Personality disorders EFFECT AND USABILITY/UX 1 100%
Major and Mild Neurocognitive Disorders NO ASSESSMENT 1 100%

NEW TABLE. Breakdown of treatment modalities which show psychological framework per study/app

NOTE: In respond to the comment F.2 ‘Breakdown of treatment modalities’

Figure 4: Distribution of articles published for the top 6 mental disorders over time.

A line chart to show the number of papers per mental disorder and year.

Table 1: Mental disorders and the studies targeting them: NA = app name not available/not mentioned.

A tabular, compact distribution of apps per mental disorder, grouping the references tha point the same app together. The number(s) in brackets next to the app name is the reference(s) in which the app is mentioned. In the table below, these references are internal identifiers instead.

Compact distribution of papers (apps) per mental disorder ‘NA’ is app not specified/available
Mental Disorder References by app
Major and Mild Neurocognitive Disorders Rico (452)
Personality disorders EMOTEO (106)
Sleep-wake disorders Sleepcare (181), SleepIO (094;103)
Feeding and eating disorders Jorvie (265), Recovery Record (460), RR: Eating Disorder Management (092), Student Bodies–Eating Disorders (SB-ED) (007)
Neurodevelopmental disorders iCanLearn (125), LifePal (142), My MFG (304), TimeOut (454)
Obsessive-Compulsive and related disorders Geo-Feedback App (262), GGOC (473), Live OCD Free (044), Mayo Clinic Anxiety Coach (056), RAW HAND (448)
Bipolar and related disorders MyT (042), PRISM (048), SIMPLe (258;226;138;459)
Comorbid disorders CASA-CHESS (297), Enuresis Trainer (290), Learn To Quit (300), SMI-CM (232), Stay Quit Coach (463), Stop-Cannabis (097), NA (086)
Suicidal behavior disorder/nonsuicidal self-injury BackUp, mEMA (453), BeyondNow (409), BRITE (470), iBobbly (124), Virtual Hope Box (VHB) (026;443), NA (275;012), Blue Ice (245;022;030)
Trauma and stressor-related disorders RELAX (301), Life Armor; PE Coach; Positive Activity Jackpot; Eventful; Tactical Breather; Virtual Hope Box; Daily Yoga; Simply Yoga (121;037), NA (306;302), PE Coach (062;061;069;049), PTSD Coach (160;118;015;088;450;478)
Schizophrenia spectrum and other psychotic disorders iCOPE (090), MindFrame (471), movisenseXS (420), RealLife Exp (419), SlowMo (260), TechCare (273), Temstem (418), Actissist (305;440), FOCUS (089;120), Heal Your Mind (087;406), PRIME (117;408)
Substance-related and addictive disorders Drink Less (468), Fit&Sobber (411), Mind the Moment (171), S-Health (437), SEVA (127), SmartQuit (217), Smoke Mind (065), Social-Local-Mobile (430), A-CHESS (013;307), CET Application (014;021), Kick.it (402;413), Smart-T (038;039)
Anxiety disorders Agoraphobia Free; Stress Free (291), Ångesthjälpen (122), AnxietyCoach (192), CBT Assistant (057), Challenger (464), Lantern (025), NOT KNOWN (457), PsychAssist (286), Public Speech Trainer (PST) (050), SmartCAT (425), GET.ON PAPP (091;105), NA (123;308;027;285;163;472)
Various disorders ACT Daily APP (110), FOCUS (414), Headspace (177), iBobbly (029), iCare-stress (433), IntelliCare Suite (131), MoodMission (427), MoodTrainer (185), myCompass (126), PeerTECH App (119), Pocket Skills (436), Sinasprite (431), SmartCAT (282), SPIRIT (140), The Moment (281), TODAY! (294), Wellframe (034), WellWave (477), NA (475), WorkingWell (220;434)
Depressive disorders 7Cups (082), Be Good to Yourself (466), BlueWatch (401), CBIA (?) (435), Dcombat (235), Get Happy Program (058), HeadGear (438), iCare Prevent (455), MedLink (152), Mobile Sensing and Support (183), MoodHacker (187), Moodivate (188), MyGamePlan (203), NOT KNOWN (432), PRIME-D (279), Push-D (407), SocioEmpathy (421), SPSRS (115), SuperBetter (231), The Sound Advice (428), Thought Challenger (114), TODAC TODAC (020), Kokoro-App (035;263;063), NA (051;023;266;071;303;077)